A. Field of the Invention
This invention relates generally to digital image processing and computer graphics. More particularly, it is concerned with generating full-sphere panorama views using subhemispherical images and simulating free movement within a multidimensional environment, which can be either computer-generated or real.
B. Description of the Related Art
Computer-generated (CG) environments are typically created by representing objects with polygons and associated computer-generated or photographic surfaces, or texture maps. Rendering, the construction of an image from a CG model, can be done from any point of view. See Foley [J. D. Foley, A. van Dam, S. K. Feiner, J. F. Hughes, Computer Graphics: principles and practice, 2nd ed., Addison-Wesley, 1987]. As such, it provides unrestricted simulated movement within the environment. However, the temporal resolution of unrestricted movement within a realistic CG environment that one can achieve on today's personal computers is severely limited by the computational requirements and by the labor of constructing realistic imagery.
U.S. Pat. No. 4,807,158 to Blanton, et. al discloses a method for reducing the computational requirements of rendering CG images, which could also be applied to natural images. First they build a database of images at selected positions, or "keypoints", within the environment by rendering them in an off-line process. They store these panoramic images as conic projections. Then in real time, the application approximates the image at any position from that at the nearest keypoint. This approach works well when all objects are about the same distance from the viewer. This is a good assumption in their application, a flight simulator, but the loss of parallax would be a severe limitation in many environments. Objects at different distances move as a unit within the domain of a keypoint, and parallax is only evident when the keypoint changes.
U.S. Pat. No. 5,396,583 to Chen, et. al captures panoramas and project them onto cylindrical surfaces for storage. They are able to rapidly project images from a cylinder to a plane using "scanline coherence". Unfortunately, like Blanton, their method does not support parallax.
McMillan, et. al. [L. McMillan and G. Bishop, Plenoptic Modeling: An Image-Based Rendering System, Siggraph '95 Proceedings, 1995], report a method that supports parallax, and apply it to natural images. They also produce a series of reference images off-line, which are captured with a video camera and re-projected to cylinders for storage. To support parallax, they calculate the image flow field between adjacent reference images. Now, when an image is approximated from a nearby reference image, different parts will move differently. Unfortunately, artifacts are quite apparent unless the image flow field is extremely accurate. Occluded regions cause additional artifacts.
The cylindrical surface (Chen and McMillan) is very inefficient for storing panoramic imagery near the vertical. Other panorama projections do not suffer from the limitations of the cylindrical projection. These include spherical, fisheye and cubic representations.
U.S. Pat. No. 5,185,667 to Zimmerman discloses a system for translating a selected portion of a hemispherical fisheye image into a planar projection for display on a CRT. The Zimmerman reference does not disclose how such hemispherical images can be seemed together by edge-to-edge abutment to form a 360 degree panoramic image. Additionally, such hemispherical fisheye images have been found to include several drawbacks that degrade the quality of the resulting translated image portion.
Accordingly, the need remains for providing improved panoramic imagry for simulating free movement within a multidimensional environment.